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ABSTRACT 



Smart Start is a broad-based community initiative to ensure 
that all of North Carolina's children arrive at school healthy and ready to 
learn. This study examined the effect of Smart Start on the quality of child 
care from 1994-1996. Data were collected from child care centers in 12 
counties implementing the initiative. Data collectors visited 180 child care 
centers in 1994 and 187 in 1996. The quality of child care was measured using 
the Early Childhood Environment Rating Scale. Findings indicated that the 
quality of child care was significantly higher in 1996 than 1994, across both 
the entire sample and the subset of 91 centers observed both years. The 
quality of child care in 1996 was significantly related to the level of 
participation in local quality improvement activities by the child care 
centers, and to the percent of full-funding counties received and the 
proportion of this funding that the county chose to spend on child care. 

These latter two variables interacted such that the proportion spent on child 
care was significantly more related to ECERS quality in the counties that 
received a higher percentage of their full -funding amount. The data indicate 
that child care was significantly improved after 2 years of Smart Start 
implementation and that factors associated with program participation were 
significantly related to the change. (JPB) 
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Abstract 



This study examined the effect of a broad-based community initiative (Smart 
Start) to improve the quality of child care between 1994 and 1996. Data 
were collected from child care centers in 1 2 counties implementing the 
community initiative. Data collectors visited 180 child care centers in 1994 
and 187 in 1996. The quality of child care was measured by the Early 
Childhood Environment Rating Scale (ECERS, Harms & Clifford, 1980). 

The quality of child care was significantly higher in 1996 than 1994, both 
across the entire sample and the subset of 91 centers observed in both years. 
The quality of child care in 1996 was significantly related to the level of 
participation in local quality improvement activities by the child care centers, 
and to the percent of full-funding counties received and the proportion of this 
funding that the county chose to spend on child care. These latter two 
variables interacted such that the proportion spent on child care was 
significantly more related to ECERS quality in the counties that received a 
higher percentage of their full-funding amount. 

These data indicate that child care quality was significantly better in the 12 
counties after 2 years of Smart Start implementation and that factors 
associated with Smart Start participation were significantly related to the 
change. This broad-based community initiative is accomplishing one of its 
major goals — improving the quality of center-based child care. 

Introduction 

The North Carolina Early Childhood Initiative, known as Smart Start, was 
established by Governor Jim Hunt in 1993 as a partnership between state 
government and local leaders, service providers, and families to better serve 
children under six and their families. The primary goal of Smart Start is to 
ensure that all children enter school healthy and prepared to succeed. 
Research in early childhood education has demonstrated the importance of 
high quality child care in preparing preschoolers for school success. 
Specifically, young children who receive high quality child care demonstrate 



better cognitive and social skills than children who receive lower c[uality 
child care (Bryant, Burchinal, Lau, & Sparling, 1994; Cost, Quality, & 
Outcomes Study, 1995; Howes, Phillips, & Whitebook, 1992; NICHD Early 
Child Care Research Network, 1997). Unfortunately, many children in North 
Carolina and across the country — do not receive high quality care. 

Smart Start's innovative approach requires local community partnerships to 
plan how best to meet their own community's needs, improve and expand 
previous programs for children and families, and design and implement new 
programs. Twelve county partnerships were competitively selected in 1993 
for a year of planning. (One partnership was actually comprised of a 7- 
county confederation, but we considered this partnership’s data as if it were 
from one county.) Between 1994 and 1996 these 12 partnerships received 
over $60 million from the NC legislature to deliver new or improved 
services. (Each year since, 12 new partnerships have been funded, but this 
report covers only the first partnerships.) 

As an important step in preparing children for school success, all local Smart 
Start partnerships funded projects in their communities to improve the quality 
of early childhood education, including center-based care. Examples of local 
projects include increased, improved, or specialized training for child care 
providers; quality improvement grants for centers to purchase educational 
curricula, equipment, and materials; and financial incentives for centers to 
demonstrate their provision of higher quality care by becoming licensed at 
the AA level (instead of A) or by achieving accreditation from the National 
Association for the Education of Young Children (NAEYC). The number of 
different quality improvement activities in which a child care center 
participated might be related to the quality of child care that the center 
provides. 

Two additional factors may affect the potential influence of these 
improvement efforts within counties: (a) the percentage of full-funding 
received by the county, and (b) the proportion of funding allocated to child 
care quality enhancement. These county-level variables are described more 
fully below. 
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(a) Percentage of Full Funding Received 

The full-funding allocation for each partnership was estimated in 1993 to be 
the amount of funds needed to improve the quality of care for children 
already in subsidized care and to enroll the rest of the county's poor children 
(birth to five) in a child care program for a half-day. The amount allocated 
each year by the legislature has fallen short of the full-funding amount. The 
percent of full-funding received may affect the level of implementation and 
success of the initiative in different counties. 

(b) Proportion spent on child care 

The local partnerships determined their own county's needs for services for 
young children and made funding decisions accordingly. Some partnerships 
allocated more of their funds to child care because they perceived a high need 
for more and better child care in their community; other partnerships chose 
instead to spend relatively more on improving health care services or 
establishing family resource or parenting education programs. The 
proportion spent on child care quality improvement activities might be 
related to the number or type of opportunities provided for child care centers 
and thus to quality enhancement. 

Smart Start has generated increased attention to early childhood education 
and child care in North Carolina, a state that has among the least stringent 
child care licensing standards. The question investigated in this paper is 
whether this type of broad-based community initiative will affect the quality 
of preschool child care. Researchers at the Frank Porter Graham Child 
Development Center collected data in 1994 and 1996 to begin answering this 
question. The main hypotheses were: 

• Overall quality of child care for preschoolers will be better in 1996 than in 
1994. 

• Preschool child care quality will be higher in counties that received more 
of their full-funding allocation. 
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• Preschool child care quality will be higher in counties that spent a higher 
proportion of their funds to improve child care. 

• Child care centers that participated in more Smart Start quality 
improvement efforts will be rated better in 1996 than those who 
participated in fewer. 
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Method 



Sample 

In 1994, researchers visited 180 child care centers in the first 12 partnerships. 
In 1996, 187 child care centers from the same counties were visited. Ninety- 
one (91) centers were visited in both 1994 and 1996. Of the centers invited 
to participate in the study, 75% agreed to do so in 1994; 64% in 1996. 

In each year of data collection, data were obtained from two samples of child 
care centers: a partnership-nominated sample and a random sample. The 
nominated sample consisted of child care centers that the 12 partnerships 
noted were involved in local Smart Start child care quality improvement 
efforts. These centers were visited in 1994 and again in 1996. The 
nominated sample was included to study directly the effect of Smart Start on 
child care in centers that were confirmed to be participating. The second 
sample of centers was randomly selected from the 1994 and 1996 lists of 
licensed child care centers in the counties (regardless of a center's 
participation in Smart Start). The random sample was included to measure 
the overall quality of care and to provide a comparison with the nominated 
sample. This process resulted in the selection of some centers both randomly 
and by nomination, a more frequent occurrence in small counties with fewer 
child care centers. In analyses, the data from such centers were included in 
both the nominated and random group. These two samples were not 
significantly different on any child care variable in 1994 or 1996, so they are 
combined in all further analyses presented here. 

Table 1 describes characteristics of the 1994 and 1996 samples, which were 
very similar on several structural characteristics of child care. 
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Table 1. Center Characteristics in the 1994 and 1996 Child Care Samples 



1994 1996 

(N=180) (N=187) 



Sample Type 



Randomly Selected Only 


83 


107 


Nominated Only 


52 


49 


Randomly Selected & Nominated 


45 


28 


Type of Center 


Not for Profit 


57% 


58% 


Church-Sponsored 


21% 


21% 


Head Start 


11% 


15% 


Independent 


48% 


44% 


Public Preschool 


4% 


2% 


Franchise 


2% 


4% 


Median % of Subsidized 


Children per Center 


38% 


41% 


Center Director with a BA 


Degree or Higher 


39% 


41% 


Lead Teachers with a BA 


Degree or Higher 


17% 


21% 


Participation in at Least 


1 Smart Start Activity 


95% 


94% 


Mean Number of Activities 


5.3 


5.9 
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Procedures 



At each center visited, data collectors completed the Early Childhood 
Environment Rating Scale (ECERS, Harms & Clifford, 1980) in one 
randomly selected preschool classroom. The ECERS is a well-established 
measure of child care quality that assesses seven general areas: personal care 
routines, furnishings and display for children, language-reasoning 
experiences, fine and gross motor activities, creative activities, social 
development, and adult needs. Scores on each of 37 items can range from I 
to 7 with the overall mean score obtained by averaging all items typically 
used as a global measure of the developmental appropriateness or quality of 
the classroom. An overall score from 1 to 3 is considered poor; scores from 
3 to 5 are considered mediocre; and scores of 5 or greater are considered 
good. 

Data collectors were trained on the ECERS to an agreement criterion of 85%, 
counting two ratings that were identical or within one point as agreements. 

In 1994, field reliability data were obtained during one visit for observers 
who rated more than 10 classrooms. These reliabilities averaged 86% 
(ranging from 75% to 92%). In 1996, field reliability data were gathered on 
each observer after every 5-8 child care visits. These reliabilities averaged 
85% (ranging from 72% to 94%). 

Data collectors also interviewed center directors to obtain information about 
center characteristics and services, including a checklist of 14 different Smart 
Start improvement activities the center or center staff might have participated 
in during the past year. The data collector was unaware of the number of 
such activities in which the center had participated because the interview was 
typically conducted after the observation. In addition, only 2% of centers 
were visited by the same data collector in 1994 and 1996. About half of the 
1996 visits were made by new observers who had not collected any 1994 data 
and the observers who gathered data both years were shifted to different 
counties in 1996. These procedures greatly reduced the possibility that any 
data collector bias influenced the results. 




Results 



Figures 1 and 2 present the distributions of the county-level predictor 
variables used in the first analyses. Figure 1 arrays the partnership counties 
by the percentage of full-funding they received. Nine counties received 
above 70% of full funding, but three others (among the most populated 
counties) received 20%, 26%, and 53% of their estimated full funding. 

Figure 2 arrays the counties by the proportion of their funding spent on child 
care quality enhancement activities. The proportion of funding devoted to 
child care quality enhancement ranged from 18% to 73%. 

These two variables and time (1994-1996) were included in a Hierarchical 
Linear Model analyses (HLM) to test the first three hypotheses. Results are 
presented in Table 2. The quality of child care as measured by the ECERS 
was significantly higher in 1996 than in 1994, E(l, 351) = 22.4, p < .0001. 
The mean ECERS score in 1994 was 4.25 ( SD = .64); in 1996, 4.51 ( SD = 
.68). Figure 3 presents the distribution of the quality of center-based care in 
1994 and 1996, illustrating a shift to the higher scores. Overall, only 14% of 
the preschool classes in 1994 were providing good quality care. In 1996, 
25% of the preschool classes were providing good quality care. 

Other evidence for increases in quality care came from the 91 centers that 
were observed in both 1994 and 1996. Among these centers quality of care 
improved significantly over the two years, E (1, 176) = 12.05, p = .0007, and 
the percentage licensed at the higher AA level increased from 38% to 52%, a 
statistically significant increase {x adjusted for repeated measures = 12.53, p 
<. 001 ). 
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Table 2 also shows that the percentage of full funding received by a county 
and the proportion spent on child care activities were each significantly 
related to quality, although these main effects should not be interpreted 
because a significant interaction was found between these variables. This 
interaction indicates that proportion of funding spent on child care accounted 
for much more of the variance in ECERS quality scores in counties that 
received a high percentage of funding compared to those that received a low 
percentage of the full-funding allocation, E (1 , 353) = 4.8 1 , p = .029. The 
effect on quality of proportion spent on child care in the low-funded counties 
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Figure 1. Distribution of Full Funding 
Received by Partnerships 
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Table 2. HLM Analyses: Child Care Quality (ECERS total score) as a 
Function of Year, Funding Level, and Proportion of Funds Spent on Child 
Care^ 





B 


se 


F 


P 


Intercept 


4.26 


.06 






Method*^ 






1.44 


.24 


Both 


.16 


.10 






Nominated 


.04 


.08 






Random 


- 


- 






Year 






21.4 


<.0001 


1994 


- 


- 






1996 


.23 


.05 






Funding Level 






3.73 


.054 


Low Funding 


-.16 


.08 






High Funding 


- 


- 






Prop. Spent 






8.33 


.004 


Low Prop. 


- 


- 






High Prop. 


1.61 


.52 






Funding Level x 






4.81 


.03 


Proportion 










Spent 










Low Funding: 


.23 


.37 






Prop. Spent 










High Funding: 


1.61 


.52 






Prop. Spent 











^Partnership and child care center were entered as random variables to adjust 
for their effects as repeated measures. 

Selection method was used as a control variable. 
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Figure 3: Quality of Preschool Classrooms 

in 1994 and 1996 
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Quality Ratings 



is much lower than in the higher- funded counties (B= .23 vs. 1.61). This 
interaction is illustrated in Figure 4 which shows that proportion of funds 
spent on child care quality improvement efforts was more strongly related to 
ECERS quality for the counties that received greater than 70% of their full- 
funding allocation. 

Figures 5 and 6 present the distribution of centers based on their level of 
participation in Smart Start child care quality improvement activities in 1 994 
and 1996. In 1994 the number of activities averaged 5.3 ( SD = 3.2) with a 
range of 0-13; in 1996, the mean was 5.9 ( SD = 3.4) with a range of 0-14. 
Using these variables, an HLM analysis was used to test the fourth 
hypothesis. Results are presented in Table 3. This analysis looked at the 
effect of an individual center’s participation in Smart Start funded child care 
efforts on the preschool quality in that center. Again there was a significant 
effect of year, indicating that 1996 quality was higher than 1994 quality. 
Participation in Smart Start quality improvement activities was also 
statistically significantly related to quality, E (1, 335) = 9.84, p = .0019, with 
centers participating in more activities likely to score higher on the ECERS. 

Simple correlations also add support to the fourth hypothesis. In the 91 child 
care centers that were observed in both 1 994 and 1 996, participation in early 
Smart Start activities was significantly related to quality of care provided in 
1996 (r = .24, p = .019). In the 1996 total sample of 187 centers, reported 
participation was also significantly positively related to the quality of care 
(r = .24,p=.001). 
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yure 4: Child Care Quality by Partnership's Proportion of Funds Spent 

on Child Care and Percent of Full Funding 
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Figure 5: 1994 Smart Start Activity Participation 
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Figure 6: 1996 Smart Start Activity Participation 
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Table 3: HLM Analyses: Child Care Quality (ECERS total score) 

Function of Year and Participation in Smart Start Activities 





B 


se 


F 


P 


Intercept 

Method^ 


4.04 


.07 


.10 


.91 


Both 


-.02 


.10 






Nominated 


-.04 


.08 






Random 

Year 


- 


- 


15.58 


.0001 


1994 


- 


- 






1996 


.21 


.05 






Participation 


.03 


.01 


9.84 


.0019 



^Selection method was used as a control variable. 
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Discussion 



The significant change over time in the observed child care quality ratings is 
a positive sign that the variety of different child care quality improvement 
efforts being implemented by Smart Start seem to be improving quality, as 
intended. The effect size (.58) is considered to be in the moderate range. 
Finding a difference of this magnitude is particularly notable since, in Smart 
Start’s first years, it has been a diverse set of "treatments" implemented in a 
wide variety of settings with a varying degree of intensity. Support for the 
conclusion that the changes seen from 1994 to 1996 were related to Smart 
Start and not just general improvement in the state is that the 1996 quality 
ratings were significantly related to three factors that theoretically should 
affect quality--the number of activities in which a center participated, the 
percent of full-funding allocation received by the county, and the proportion 
of funds allocated to child care. Further, the rate of increase in the proportion 
of centers licensed at the AA level has been higher in Smart Start counties 
than in other North Carolina counties. 



The finding that the proportion of funding spent on child care was 
significantly related to improvements in quality indicates that focusing fiscal 
resources in a targeted area (i.e., child care quality improvement) while 
allowing counties to decide how resources are spent is a viable strategy to 
improve overall quality of care. It is not surprising that this influence was 
much stronger in counties that received a high proportion of their full- 
funding allocation compared to low-funded counties. For example, the 
county that spent the highest proportion of its allocation on child care was 
also the county that received the lowest percentage of its full-funding 
allocation. Under these conditions, the effect of a high proportion of child 
care spending is less. 



A higher percentage of centers refused to participate in the study in 1 996, 
which the directors sometimes attributed to having participated in too much 
research recently (indeed true in some Smart Start counties) and sometimes 
to a dissatisfaction with the local Smart Start decision-making. Although it is 
possible that more centers of lower quality refused participation in 1996 than 
in 1994, significant improvements in quality occurred in the sample of 91 
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centers seen in both years. In addition, the significant relationships between 
predictors (level of funding, proportion spent on child care, participation in 
improvement activities) and outcome (improved ECERS) existed regardless 
of selective refusal, adding to our confidence in these findings. 

We should note that these findings pertain to the quality of preschool classes 
for children in North Carolina’s first 12 Smart Start counties, not to the 
quality of infant and toddler care. Other studies have shown that infants 
generally receive less safe and developmentally appropriate care in group 
settings than do preschoolers (Cost, Quality, and Outcomes Study, 1995). 
When we began this study we intended to include a sample of infant care, but 
found that very few centers provided care for infants and few Smart Start 
activities were directed specifically to improving the quality of infant care. 

As more Smart Start activities become directed at infant care, a more focused 
study of infant care might be desirable. 

One of the most interesting findings was the large number of centers that 
were indeed participating in Smart Start- funded quality improvement efforts. 
Many centers took advantage of multiple opportunities. We expected this in 
the nominated sample, but it was also true in the random sample, which is 
probably why we found no difference in samples recruited in these two 
different ways. Smart Start is reaching a large number of centers in counties 
large and small, urban and rural. Its effect can be most noted to date in the 
increase in child care quality from 1994 to 1996 and in the significant 
relationship between participation in Smart Start and observed quality of 
care. 

A second finding of note was the relatively large number of children from 
poor families being served by the centers. About 40% of the children in the 
hundreds of centers observed were receiving a child care subsidy. (Full 
subsidies are usually given to the children of unemployed poor families with 
smaller subsidies provided for children as parents move up the income scale.) 
It appears that centers benefiting from the Smart Start quality improvement 
efforts serve families in a range of incomes, thus benefiting a wide range of 
children, not just those from a single income group. 
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This study does not answer questions about child care quality improvement 
activities that many will want to know. Which types of activities are most 
effective in improving preschool classroom quality? Are in-service 
workshops more effective than sending teachers to community colleges for 
further training? Are enhancement funds better spent on literacy materials 
than on playgrounds? The Smart Start evaluation is not a randomized study 
that can address these questions. Some counties did not offer all 14 different 
types of quality enhancement activities, and child care centers within a 
county chose to participate in as many or as few activities as they desired or 
were allowed. Because centers vary in their own starting points and needs, it 
is likely that the best and most effective improvement activities for one center 
would be somewhat different than those that would benefit another center. 
Our data do show, however, that more participation is related to increased 
preschool classroom quality. 

In conclusion, the effect of North Carolina's commitment to young children 
and their families as evidenced by legislative and community support and 
funding for the Smart Start program is now being seen in improved quality of 
child care for preschoolers. The evaluation of this initiative will continue to 
include monitoring of child care quality as well as changes in child health 
and readiness, family services, and collaboration among agencies serving 
children and families. 
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